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ABSTRACT 

Over the past decade, a new kind of entity has emerged in 
public education: the for-profit provider of education and management 
services. This is a report on a study of the effectiveness of the programs of 
three such companies — Edison Schools, Mosaica Education, and Chancellor 
Beacon Academies — operating in 10 charter schools in the District of 
Columbia. The study set out to: (1) identify the characteristics of the 

companies operating in the district and determine the extent to which schools 
managed by these companies have used their programs; and (2) determine what 
is known about the effectiveness of these programs as measured primarily by 
student achievement. Data were gathered from company websites; reviews and 
summaries of published and unpublished research; observations; and interviews 
with company officials, representatives from the 10 schools, and officials of 
the district’s chartering authorities. Results show that the extent to which 
the 10 schools implemented the elements of the companies’ programs varied. 
Additionally, little is known about the effectiveness of these companies’ 
programs on student achievement, parental satisfaction, or school climate. 
This report concludes there is not enough evidence to determine the 
effectiveness of the. programs offered by the three companies. (Contains one 
table and four references.) (WFA) 
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^ G A O 

Accountability * Integrity * Reliability 

United States General Accounting Office 
Washington, DC 20548 



October 29, 2002 

The Honorable Chaka Fattah 
Ranking Minority Member 
Subcommittee on the District of Columbia 
Committee on Appropriations 
House of Representatives 

Dear Mr. Fattah; 

In recent years, local school districts and traditional public schools have 
taken various initiatives to improve failing schools. For example, school 
districts and charter schools are increasingly contracting Avith private, for- 
profit companies to provide a range of education and management 
services to schools. To date, there has been debate regarding the 
effectiveness of such companies in managing public schools. 

These companies generally offer schools services in areas such as school 
organization, instruction, technology, and professional development. In 
the District of Columbia, some public schools currently contract with 
three such companies; Edison Schools, Mosaica Education, and 
Chancellor Beacon Academies. As agreed with your office, we 
(1) identified the characteristics of their programs and determined the 
extent to which District schools managed by them have used their 
programs and (2) determined what is known about the effectiveness of 
these companies’ educational programs, as measured primarily by student 
achievement. 

To address these issues, we reviewed relevant research on charter and 
traditional public schools managed by for-profit educational management 
companies as well as documents and materials provided by the 
companies. In addition, we observed an on-site review of one school’s 
program which was conducted for District oversight authorities. We also 
interviewed officials of the companies that manage District public schools, 
officials of the District’s oversight authorities, and representatives of the 
schools, as well as officials of the Department of Education, other 
education experts and advocates. Finally, we reviewed evaluations 
concerning the three companies operating in the District that met the 
following criteria; included comparison groups and measurement of 
performance over time, and focused on academic achievement, parental 
satisfaction, parental involvement, or school climate. We assessed the 
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Results in Brief 



o 

ERIC 



quality of the evaluations’ research designs, reviewed them for threats to 
validity and determined whether we had confidence in their conclusions. 
We conducted our work between January and September 2002 in 
accordance with generally accepted government auditing standards. 



Edison, Mosaica, and Chancellor Beacon have programs that consist of 
both management services, such as personnel, and educational services, 
which they offer to schools across the nation; in the District, most of the 
schools managed by these companies have either adopted selected 
elements of their companies’ programs or chosen other educational 
programs. Each company provides services such as curriculum, 
assessments, parental involvement opportunities, and student and family 
support. They also offer a variety of organizational options, including 
smaller class and school sizes, as well as longer school days and school 
years. All of the companies allow their schools some flexibility in adapting 
their programs to local circumstances. The extent to which the District 
schools implemented all the elements of these companies’ educational 
programs varied. For example, 6 of the 10 schools managed by these 
companies had either partially implemented the company’s curriculum or 
had not implemented that curriculum at all. Some schools have opted to 
customize the company’s educational program; other schools have left in 
place the educational program of a company that formerly managed them. 
In school year 2001-02, all 10 District schools managed by these companies 
were charter schools with predominantly poor and minority student 
populations; most enrolled elementary and middle school students. 



Little is known about the effectiveness of these companies’ programs on 
student achievement, parental satisfaction, parental involvement, or 
school climate because few rigorous studies have been conducted. While 
the companies publish year-to-year comparisons of standardized test 
scores to indicate that students in schools they manage are making 
aca;demic gains, they do not present data on comparable students who are 
not in their programs, a necessary component of a program effectiveness 
study. An effectiveness study attempts to isolate the effect a program has 
on specific outcomes by, among other things, comparing outcomes for 
students in company managed schools with those of a comparable group 
of traditional public school students, tracking students over time, testing 
students before and after exposure to the company’s program, and 
controlling for differences between these groups. Of five studies we 
identified concerning the three companies operating in the District, four 
focused only on one company. Moreover, of the five studies, one — based 
on one school in Florida — was rigorous enough to allow confidence in the 
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findings about the program’s effectiveness in that school. This particular 
study, using two analytical techniques, found no difference between 
students in the company’s program and other students. The remaining 
studies had methodological limitations that precluded assessments about a 
company’s effect on student achievement, parental satisfaction, parental 
involvement, and school climate. Additional research on one company’s 
program is planned by an organization with experience in conducting 
educational evaluations. 



Background 



Dtuing the last decade, a new kind of entity has emerged in public 
education: the for-profit provider of education and management services. 
Historically, school districts have contracted with private companies for 
noninstructional services, such as transportation and food service, and 
have also relied on contractors in some cases to provide limited 
instructional services to specified populations. Until recently, public 
schools have generally not contracted for the comprehensive programs of 
educational and management services that these companies typically 
offer. In recent years, the options available to public schools considering 
contracting with private companies have steadily grown. Today, 
approximately 20 mgjor companies manage public schools.* Nationally, it 
is estimated that these companies as weU as other smaller companies 
serve over 300 schools out of the nation’s approximately 92,000 public 
schools. Although these companies manage public schools at all grade 
levels, most such privately managed public schools are elementary and 
middle schools. In these public schools, companies generally provide the 
same kinds of educational and management services that school districts 
do for traditional public schools. Educational services typically include a 
curriculum as weU as a range of services designed to enhance or support 
student achievement, such as professional development opportunities for 
teachers, opportunities for parental involvement and school environments 
that aim to facilitate student support. Management services typically 
include personnel, payroll, and facilities management. Although these are 
the services that are typically offered to schools, companies also may 
adapt their services to respond to the preferences or needs of individual 
schools. For example, while some companies offer a particular curriculum 



'in this context, “m£yor” refers to those companies that manage at least 3 schools and 
operate in multiple states. 
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or educational approach, others appear more willing to work with the 
curriculum the school or school district has already adopted.^ 

Typically, companies provide their services to public schools in one of two 
ways. First, they can contract directly with school districts to manage 
traditional public schools; such schools are known as “contract schools.” 
Second, they can manage charter schools, which are public schools that 
receive a degree of autonomy and freedom from certain school district 
requirements in exchange for enhanced accountability. Generally, charter 
schools are run by individual boards of trustees, which in most states and 
the District of Columbia have the authority to decide whether to contract 
with a private company.^ Both contract schools and charter schools 
remain public schools, however, and are generally subject to federal and 
state requirements for public schools in areas such as the application of 
standardized tests and special education. 

While the reasons public schools turn to private companies vary, the 
potential to increase student achievement appears to be one factor. In 
particular, according to certain experts and company officials we spoke 
to, school districts that seek a company’s help often do so with the 
expectation of raising achievement in struggling or failing schools. While 
management services appear to be especially important for charter 
schools that contract with such companies, charter schools also consider 
the potential to raise student achievement or a particular educational 
approach consistent with the school’s mission, according to school 
officials and experts we spoke with. Both types of schools that seek these 
companies’ assistance — struggling schools and charter schools — appear 
concentrated in urban areas. Further, several of the mqjor companies 
reportedly serve a predominantly disadvantaged urban and minority 
student population. 

Recent changes in federal law have implications for the role played by 
these companies in public schools. The No Child Left Behind Act of 200F 
requires that schools that fail to meet state student achievement standards 
for 6 consecutive years must be restructured by implementing one or more 



In fact, not all companies offer a particular curriculum or educational approach, focusing 
instead on management services. For example, Arizona-based ABS concentrates on 
management services like recruitment, personnel, payroll, and facilities management 

however, in Arizona, private companies may run charter schools directly. 

'RL. 107-110, Jan. 8, 2002. 
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alternative governance actions. One of the alternatives available to states 
and districts is to contract with an education management company.^ 

Three companies currently operate in the District of Columbia: Edison 
Schools, Mosaica Education, and Chancellor Beacon Academies. Edison 
began operating its first District school in 1998, and Mosaica and 
Chancellor Beacon first contracted with the District schools they manage 
in 2001. Throughout this report, these companies will generally be 
discussed in this order. 

Mergers and acquisitions are common among such companies. In 2001, 
Edison acquired nine schools nationwide through a merger with 
LeamNow. In the same year, Mosaica acquired nine schools nationwide 
through its acquisition of Advantage Schools. In addition. Chancellor and 
Beacon merged into a single company. Such changes can have several 
outcomes: in some cases, the company may operate schools that continue 
to use the educational program of another company; in other cases, the 
school may consider adopting the educational program of the new 
company or terminating the contract. 



Education 
Management 
Companies Have 
Multifaceted 
Programs; District 
Schools Varied in 
Their Implementation 



The companies that operate public schools in the District of Columbia 
offer management and educational services as part of their programs; the 
extent to which District schools managed by these companies 
implemented aU of the components of the companies’ programs varied. All 
of these companies offer programs that include management and 
educational services, such as curricula that integrate technology and 
professional development opportunities for teachers. Of the 10 District 
schools managed by these companies, 4 had completely implemented their 
company’s program. In school year 2001-02, all 10 District schools 
managed by these companies were charter schools with predominantly 
poor and minority student populations; most enrolled elementary and 
middle school students. Similar to traditional public schools, the District 
schools managed by these companies were required to be open to all 
students, up to their enrollment limits, and to meet District standards in 
areas such as health, safety, standardized testing, and compliance with 
federal special education requirements. 



^Others include reconstituting by replacing teachers, principals, or both; converting to 
pubUc charter school status; and turning over school operation to the state educational 
agency. 
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All Three Companies Offer 
Management and 
Educational Services to 
Schools Nationally 



The three for-profit companies that operate in the District of Columbia® — 
Edison, Mosaica, and Chancellor Beacon — share common elements in 
terms of the management and educational services they offer to schools 
nationwide as well as those company officials described as distinctive. 
Each of the three companies generally offers similar management services. 
For example, all three offer management services such as personnel, 
payroll and facilities management, services that can be important for 
charter schools. In addition, the three companies employ some common 
approaches designed to improve student achievement. All three 
companies offer an extended school day and year. All three integrate 
technology in their educational programs. For example, all three offer 
students access to classroom computers. Similarly, all organize schools 
into smaller units to facilitate their tracking of students’ progress. All three 
provide summer training to teachers as well as other forms of professional 
development. Additionally, all have activities designed to involve and 
support parents and students. For example, each company uses parent 
satisfaction surveys. Experts we spoke to noted that these same 
approaches were being used in some other public schools. Finally, 
officials of all three companies stated that their companies contributed 
positively to school climate — a sense of mission and an environment 
conducive to learning — and cited aspects of school climate such as a safe 
and orderly school environment and teacher motivation. In addition to the 
characteristics they had in common, company officials identified others 
they believed were distinctive. These include, for example, their programs’ 
curriculum and instruction as well as the ability to provide economies of 
scale, develop community partnerships, and provide strong administrative 
support. As Table 1 shows, all three companies provided their services to 
schools in multiple states in 2001-02. 



®In school year 2001-02, a fourth for-profit company, Richard Milbum Academies, 
continued to operate a District public charter school, pending a final decision by District 
oversight authorities to revoke its charter. The school closed in June 2002. 
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Table 1 : Profiles of Three Education Management Companies Operating in the 
District of Columbia 





Edison 


Mosaics 


Chancellor 

Beacon 


Year company was established 


1992 


1997 


2oor 


Number of states^ where company operated 
schools in school year 2001-02 


24 


8 


11 


Number of schools company operated 
nationwide in school year 2001-02 


136 


21 


82 


Number of schools company operated in the 
District of Columbia in school year 2001 -02 


6 


2 


2 



'Year of merger of Chancellor Academies (originally founded in 1999) and Beacon Education 
Management (founded in 1993). 

'’Including the District of Columbia. 

Source: Edison, Mosaics, and Chancellor Beacon. 

According to Edison officials, its program has a number of distinctive 
characteristics.^ The first of these is its curriculum, which emphasizes 
basic skills, especially reading as the basis for future learning. It also 
includes enrichment in areas such as world languages (e.g., Spanish) and 
art. Edison’s basic skills curriculum includes components developed by 
Edison, such as a remedial reading program, and other components that 
Edison states are supported by research, such as Chicago Math and the 
Success for AU reading program. Instructional methods are a second 
characteristic of Edison’s program. Edison schools use a variety of 
instructional methods. One of these, direct instruction, relies on repetition 
and drill. Other methods use projects, small groups, and individualized 
lessons. A third characteristic of Edison schools is their use of 
assessments. According to Edison officials, their program uses frequent 
assessments and the results of these assessments are promptly provided to 
teachers to assess student needs and provide appropriate additional help. 
“Systems and scale” is another key characteristic of Edison schools 
according to company officials. The company views its schools as part of a 
national system linked by a common purpose, and because of the system’s 
size, the company says it is able to purchase supplies at lower costs. 



^Edison officials describe the educational approach as consisting of “10 fundamental” 
characteristics: school organization, extended school day and year, curriculum, 
instructional methods, assessments, professional teaching environment (including 
professional development), use of technology, partnership with families, instructional 
programs that reflect community interests, and the benefits of “systems and scale.” 
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Mosaica officials also identified certain distinctive characteristics of their 
company’s program.® The first is the program’s curriculum, which has two 
parts. According to Mosaica officials, its morning program features 
instruction in traditional subjects such as reading and math. In the 
afternoon, students use Paragon — Mosaica’s own curriculum. According • 
to company officials. Paragon stresses multidisciplinary learning, uses 
projects to emphasize the humanities, and recognizes students’ different 
learning styles. For example, students may use their reading, math, and 
social studies learning to build a pyramid or a Viking ship and thus study a 
period of history. According to company officials, projects accommodate a 
variety of learning styles — for example, some students learn visually, 
others by performing. Community involvement is a second key 
characteristic of Mosaica’s program. Company officials say that Mosaica 
brings community support into the school by networking with various 
commimity organizations. According to company officials, this provides its 
schools with access to additional resources. 

Chancellor Beacon officials also identified distinctive characteristics of 
their program. One is their willingness to customize their educational 
program to meet the needs and preferences of local schools. For example, 
in response to community interest, some Chancellor Beacon schools 
feature a cultural heritage element in the curriculum while one of its 
schools emphasizes the environment. Chancellor Beacon’s OAvn 
curriculum was recently finalized in July 2002 and is based on an 
integration of the curricula of Chancellor and Beacon before they merged. 
One component of its curriculum is Core Knowledge — a program that 
expects students to master specific content in language arts, history, 
geography, math, science and fine arts. Other components emphasize 
ethics, morality and community volunteerism. A second key characteristic 
of Chancellor Beacon’s program is its operational support, according to 
company officials. These officials told us that in focusing on operational 
support. Chancellor Beacon allows schools to focus on academics. 

While the Chancellor Beacon program emphasizes customization as a key 
characteristic, the other two companies also allow schools to modify their 
programs. For example, in its reading program, Edison allows schools 
some flexibility regarding what books to read and in what order. In 



®Mosaica officials describe the educational approach as consisting of “seven pillars” or 
characteristics: the Paragon curriculum, professional development, use of technology, 
extended school day and year, parental involvement, community involvement, and school 
climate. 
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addition, up to one-fourth of its curriculum can be determined by the local 
school. Similarly, Mosaica allows its schools to use different approaches 
or materials in their morning session. 



Of the 10 District Schools, 
4 Completely Implemented 
the Company’s 
Educational Program 
While 6 Schools Selected 
Elements of Their 
Companies’ Programs or 
Chose Other Programs 



While all of the 10 District schools managed by the companies during the 
2001-02 school year obtained management services from these companies, 
the schools were more selective in implementing the companies’ 
educational programs. Of the 10 District schools, 4 have completely 
implemented the companies’ educational programs and 6 have adopted 
selected elements of their companies’ programs or chosen other programs, 
typically those of a previous company. A key factor that helps explain the 
difference between the programs the companies offer and what has been 
implemented by District schools is that recent mergers and acquisitions 
have led to changes in management companies in these 6 schools; these 
schools have generally left in place the educational programs of the 
companies that formerly managed them. 



Four schools, all managed by Edison, implemented the company’s 
educational program completely, according to company officials. These 
4 schools all opened in 1998 as the result of a partnership between 
Friendship House, a nonprofit community organization serving District 
children and youth since 1904, and Edison. According to a Friendship 
House official, these schools completely implemented Edison’s program 
because they saw it as complementing their own goals. One of these 
schools — a high school — has supplemented the Edison program by 
developing a program to expose certain students to college through 
campus visits and workshops for parents. 



Six District schools adopted selected elements of their companies’ 
educational programs or chose other educational programs. These 
6 schools include 2 schools managed by Edison, 2 by Mosaica, and 2 by 
Chancellor Beacon. All 6 schools have had recent changes in management 
companies as a result of mergers or acquisitions. 



The 2 schools that received services from Edison have opted to retain the 
curriculum already in place at the schools, rather than adopt the Edison 
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program.® In 2001, Edison bought LeamNow, the company that formerly 
provided services to the 2 schools. According to an Edison official 
knowledgeable about the schools formerly managed by LeamNow, the 
primary difference between the companies’ curricula was in elementary 
language arts, for which LeamNow preferred a different reading program 
than Success for All, which the Edison program uses in its other schools. 

The 2 schools managed by Mosaica have adopted some elements of the 
company’s educational program, and have plans to adopt more by 2003. In 
2001, Mosaica bought Advantage, the company that formerly managed 
these schools. Both schools retained an instmctional approach put in 
place by the previous company. This approach — direct instmction — 
emphasizes drill and repetition. By school year 2003, both schools expect 
to use direct instmction during the morning session and Paragon in the 
afternoon. 

The 2 schools managed by Chancellor Beacon both had distinct curricula 
in place before being managed by this company; one has combined its 
existing curriculum with elements of Chancellor Beacon’s, and the other 
has left its existing curriculum in place. The school that has adopted 
elements of Chancellor Beacon’s curriculum has done so by integrating 
the company’s language arts and math curriculum vdth the school’s 
existing curriculum, according to company officials.'® This school, which 
serves at-risk youth, had a curriculum called expeditionary learning, which 
focuses on learning through field trips and experiences. The other 
Chancellor Beacon school opted to retain its existing basic-skills 
curriculum, relying instead on the company’s management services and 
selected educational services, such as assessments. Chancellor Beacon 
officials support the schools’ choices regarding what company 
components to adopt. 

Company and school officials identified several reasons why these 
6 schools did not completely implement the current company’s 
educational program, opting instead to continue with an existing 



^According to an Edison official, as of September 2002, Edison and one of these 2 schools 
were planning a transition that would completely discontinue Edison’s services to the 
school and return the school to management by its own board of trustees. The services that 
Edison provided to both schools in the 2001-02 school year were limited to management 
services, such as payroll. 

"This school’s existing educational approach is tied to a 3-year federal comprehensive 
school reform demonstration grant. 'The grant expires at the end of the 2004 school year. 

13 
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curriculum. These included continuity for students, the company’s 
flexibility with regard to local customization, and the right of charter 
school boards to make broad curriculum decisions. “ 

The 10 schools in the District managed by these companies shared certain 
characteristics and served similar student populations in 2001-02. All were 
public charter schools governed by their own boards and accountable to 
District oversight authorities. Most (9) were combined schools spanning 
elementary and middle school grades. As public schools, they were 
required to accept any student who applied, up to their enrollment limit. 
Their student populations were substantially minority and poor: 92 to 
100 percent Aftican American and 48 to 95 percent receiving free or 
reduced school lunch.‘^ All served some students with special needs, such 
as learning disabilities: in 9 of the schools, the percentage ranged from 5 to 
13 percent, and in one school, 32 percent of the student population had 
special needs. All but one served no or very few students with limited 
English proficiency; at the remaining school, students with limited English 
proficiency represented about 12 percent of all students enrolled. 



Limited Research 
Exists of the 
Effectiveness of 
These Companies’ 
Programs 



Little rigorous research exists on the effectiveness of the three educational 
management companies — Edison, Mosaica, and Chancellor Beacon — in 
the schools they manage across the country; as a result, we cannot draw 
conclusions about the effect that these companies’ programs have on 
student achievement, parental satisfaction, parental involvement, or 
school climate. Students in company managed schools have demonstrated 
academic progress, but more research is needed to determine if this 
improvement is directly the result of the companies’ programs and if this 
progress is different from that of comparable students in traditional public 
schools. We reviewed five studies that addressed student achievement, but 
only one was conducted in a way that allows an assessment of the effect 
the company’s program had on student achievement in one school. The 
remaining studies had methodological limitations that precluded such 
assessments. In an effort to learn more about effectiveness, Edison has 
recently commissioned RAND, a nonprofit research organization that has 



addition, according to officials of the agencies that oversee District charter schools, 
substantial curriculum changes must be officially approved. 

^^Data were not available for all schools; these ranges are based on 8 schools (percent 
Afirican American) and 9 schools (percent receiving subsidized lunch). 
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evaluated educational reforms, to complete a study to assess its program’s 
impact. 

Determining the effect of an educational company’s program can be 
challenging for researchers. Ideally, evaluations of program effectiveness 
should involve a comparison of outcomes for one group exposed to a 
particular program with outcomes of a second group not exposed to the 
program. Some evaluations assign participants randomly to one group or 
the other to increase the likelihood that the two groups are roughly 
equivalent on all characteristics that could affect outcomes. 'This technique 
of random assignment is often problematic in educational research 
because public school enrollment is generally based on residency 
requirements. Therefore the most common way to compare student 
achievement results from two different groups of students is to ensure the 
groups are similar in a number of ways, including socioeconomic status, 
ethnicity, and performance on prior academic assessments. In addition to 
controlling for the effects of these background characteristics, it is critical 
to follow the performance of students over time, preferably before any 
group has been exposed to the program, and at least one point thereafter.'® 
It is also beneficial to analyze individual student data, rather than grade or 
school-level averages, to account for individual differences and to factor in 
the effects of missing data. 

Within the context of rigorous educational program evaluations, various 
measurements can be used to capture a student’s performance on 
standardized tests. According to several experts, it is important to examine 
both the percent of students in a particular grade or school making yearly 
gains and the distribution of these gains across ability levels to ensure that 
students of all achievement levels are demonstrating academic growth. 
Another point of interest relates to the length of time students participate 
in a particular program. Some experts claim that students will exhibit 
greater gains the longer they participate in a program. However, it is 
particularly challenging to design studies that address this claim, because 
educational companies are stiU a relatively new phenomenon. 

We identified five studies concerning the three companies operating in the 
District that met the criteria for our review: inclusion of comparison 



'^Experts varied with regard to how many years of data are sufficient for analysis. 
Generally, experts we spoke to indicated that 3 to 5 years of data would be sufficient for 
analysis. 
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groups, measurement over time, and focus on academic achievement, 
parental satisfaction, parental involvement, or school climate. All of the 
studies addressed the effectiveness of schools managed by Edison. One 
study also addressed the effectiveness of schools managed by all three 
private companies — Edison, Mosaica, and Chancellor Beacon."’ We were 
unable to identify any rigorous studies that included analysis of District 
public schools managed by any of these three companies.’® Of the studies 
included in our review, four studies addressed only outcomes related to 
student achievement, while one study addressed student achievement and 
other outcomes such as parental satisfaction and school climate.” 

Only one of the studies, A Longitudinal Study of Achievement Outcomes 
in a Privatized Public School: A Growth Curve Analysis, based on one 
Edison school in Miami-Dade Coimty, Florida, was conducted in a way 
that allows an assessment of the program’s effect on student achievement. 
This study followed individual student standardized test scores over a 
3-year period and found that Edison students progressed at similar rates to 
those in the traditional Miami-Dade Coimty Public Schools (MDCPS); this 
finding is not generalizable to other schools managed by Edison or any 
other private company. The study was designed to ensure that the Edison 
students were similar to the random sample of students drawn from 
MDCPS in terms of school grade, socioeconomic status, as indicated by 
the percent eligible for free/reduced price lunch, ethnicity, and 
achievement levels, as indicated by comparability in test scores prior to 
students enrolling in the Edison school. The study employed two different 
analytical techniques and both resulted in the finding that the Edison 
students progressed at similar rates to the traditional public school 



'‘'while the study, Achievement Performance Report: Dallas-Edison Pa.rtnet'ship Schools 
2001 -02 (Dallas Division of Evaluation and Accountabihty, Dallas Independent School 
District, 2002), met the criteria for inclusion, we were unable to review it because it was 
published after completion of the review. 

'®The evaluation included 18 schools managed by Beacon and was completed prior to the 
merger of Chancellor and Beacon. 

'®One study compares academic achievement in District charter schools and traditional 
pubUc schools, but does not distinguish charter schools managed by private companies. 
See Jeffrey Henig et al. Gmiving Pains: An Evaluation of Charter Schools in the Dislricl 
of Columbia; 1999-2000. (Washington, D.C.: The Center for Washington Area Studies, The 
George Washington University, 2001). 

”We did not review the student achievement portion of the Miami-Dade County Public 
Schools Evaluation because the same data were analyzed, with veiy similar results, by the 
same researcher in another study in the review. 
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students. Several methodological techniques that would have strengthened 
its overall findings could have been employed. These include controlling 
more specifically for school-level differences between the participating 
students as well as better ensuring the two groups of students remained 
equivalent despite study dropouts (subsequently referred to as attrition). 
Differences in the composition of these groups, after attrition, could affect 
the test score results. This study did not examine the effect of this 
company’s program on parental satisfaction, parental involvement, or 
school climate. 

Significant limitations in the other four studies preclude our making 
assessments of the effectiveness of schools managed by Edison, 
Chancellor Beacon, or Mosaica that were included in the studies.’® These 
limitations included use of comparison groups that did not adequately 
control for differences between the students in the company’s schools and 
the students in traditional public schools, instances where achievement 
data were not available for aU students, and lack of acljustment for high 
attrition rates. 

Company officials report that one way to determine if their programs are 
effective is to assess whether students demonstrate academic growth as 
evidenced by improvement on standardized tests. 'There is evidence to 
support the assertion that students enrolled in schools managed by 
Chancellor Beacon, Mosaica, and Edison have demonstrated academic 
improvement from one point in time to another, but it is important to 
determine if these gains are specifically the result of company programs. 

Additional research is in progress. Edison commissioned RAND to 
evaluate Edison schools across the country. Where possible, RAND plans 
to compare the scores of individual Edison students to those of traditional 
public schools students with similar characteristics. Since it is often 
difficult to gather individual level student data, RAND will also compare 
Edison data, either at the grade or school level, to publicly available state 
data at that same level. RAND expects to publish its findings in 2004. 



Agency Comments received written comments on a draft report from the Department of 

Education. These comments are presented in appendix El. Education 



'®Please see appendix II for the studies and other information sources considered for our 
review. 
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stated that there are insufficient data on the effectiveness of private 
education companies. Education also stated that it encourages others’ 
evaluation efforts. We also received comments from an expert on private 
education companies, the authors of the MDCPS study that we assessed, 
the District of Columbia Board of Education, the District of Columbia 
Public Charter School Board, as well as Edison Schools, Mosaica 
Education, and Chancellor Beacon Academies. These comments were 
also incorporated where appropriate. 



We are sending a copy of this report to the Secretary of Education, the 
District of Columbia Board of Education, the District of Columbia Public 
Charter School Board, Edison Schools, Mosaica Education, and 
Chancellor Beacon Academies. We will make copies available to others on 
request. In addition, the report will be available at no charge on the GAO 
Web site at http;//www.gao.gov. 

If you or your staff have any questions about this report, please call me at 
(202) 512-7215. Other contacts and contributors to this report are listed in 
appendix IV. 

Sincerely yours, 

Mamie S. Shaul, Director 

Education, Workforce, and Income Security Issues 
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Appendix I: Objectives, Scope, and 
Methodology 



Objectives 



The objectives of our review were to (1) identify the characteristics of the 
for-profit educational management companies operating in the District 
and determine the extent to which District schools managed by these 
companies have used their programs and (2) determine what is known 
about the effectiveness of these programs, as measured primarily by 
student achievement. We conducted our work between January and 
September 2002, in accordance with generally accepted government 
auditing standards. 



Scope and 
Methodology 



To identify the characteristics of the programs offered by for-profit 
companies operating in the District, and determine the extent to which 
District public schools managed by them have used their programs, we 
interviewed company officials, representatives of the 10 schools, as weU as 
officials of the District’s chartering authorities. We collected information 
on the companies from their Web sites and obtained technical comments 
from the companies on the descriptions of their programs. We also 
contacted education experts and advocates to obtain both their 
recommendations on research regarding the three for-profit companies 
and information on any research they might have conducted on the 
companies. We also acquired information on the companies by reviewing 
relevant research summaries. We also observed an on-site review of one 
school’s program conducted for District oversight authorities. 

To determine what is known about the effectiveness of these programs, 
we collected, reviewed, and analyzed information from available published 
and unpublished research on the effect on student achievement, parental 
satisfaction, parental involvement, and school climate of the three 
companies managing schools in the District. We also spoke with RAND 
officials about the design and methods of their current evaluation of 
Edison Schools. To identify relevant research, we followed three 
procedures: (1) interviewed experts to find out what studies were 
completed or in the process of being completed on the effectiveness of 
company programs; (2) conducted library and Internet searches; and 
(3) reviewed bibliographies of studies that focused on the effectiveness of 
company programs. 

We reviewed studies concerning the three companies operating in the 
District that met the following criteria: included comparison groups and 
measurement over time, and focused on academic achievement, parental 
satisfaction, parental involvement, or school climate. Our final list of 
studies for review consisted of five studies, as listed in appendix II. We did 
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Appendix I: Objectives, Scope, and 
Methodology 



not identify any studies that evaluated the effect of these three programs 
in District schools. 

Two GAO social scientists examined each study to assess the adequacy of 
the samples and measures employed, the reasonableness and rigor of the 
statistical techniques used to analyze them, and the validity of the results 
and conclusions that were drawn from the analyses. For selected studies, 
we contacted the researchers directly when we had questions about their 
studies. 
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Appendix II: Studies and Other Information 
Sources Considered 



In order to identify research that explicitly addresses the effect on student 
achievement, parental satisfaction, parental involvement, or school 
climate of the three companies managing schools in the District, we 
interviewed experts to determine what studies were completed or in the 
process of being completed, conducted library and Internet searches, and 
reviewed bibliographies of studies that focused on the effect of these 
companies’ programs on student achievement. Although five studies met 
our criteria for review (inclusion of comparison groups, measurement 
over time, and focus on academic achievement, parental satisfaction, 
parental involvement, or school climate), we cannot draw conclusions, 
due to methodological weaknesses, from the four studies listed below.' 
Conclusions fi"om A Longitudinal Study of Achievement Outcomes in a 
Privatized Public School: A Growth Curve Analysis were presented in the 
text. 

• Miron, Gary and Brooks Applegate. An Evaluation of Student 
Achievement in Edison Schools Opened in 1995 and 1996. Kalamazoo, 
Michigan: The Evaluation Center, Western Michigan University, December 
2000. 

• Miron and Applegate analyzed both individual and aggregate level data 
and compared improvements in the test scores of 10 Edison schools 
with those of comparison schools, districts, states, and national norms, 
where applicable. However, significant weaknesses prevented 
conclusive statements on the effects of Edison schools. These 
weaknesses included limitations in the available data, such as 
incompleteness and inconsistency, high attrition rates, and the lack of 
corresponding adjustments for attrition. 

• Horn, Jerry and Gary Miron. An Evaluation of the Michigan Charter 
School Initiative: Performance, Accountability, and Impact. Kalamazoo, 
Michigan; The Evaluation Center, Western Michigan University, July 2000. 

• Horn and Miron examined the percentage of students earning a passing 
grade on achievement tests in individual charter schools in Michigan in 
comparison with the percentage passing in the districts where these 



'while the study, Achievement Performance Report: DaUas-Edison Partnership Schools 
2001-02 (Dallas Division of Evaluation and Accountability, Dallas Independent School 
District, 2002), met the criteria for inclusion, we were unable to review it because it was 
published after completion of the review. 
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Appendix II: Studies and Other Information 
Sources Considered 



schools were located.^ The analysis included schools managed by 
Edison, Mosaica, and Beacon. Weaknesses included inadequate 
controls for differences between the students in charter schools and 
their host districts, no consideration of attrition rates, and the 
likelihood that analyses were often based on a small number of 
students. 

• American Federation of Teachers. Trends in Student Achievement for 
Edison Schools, Inc.: The Emerging Track Record. Washington, D.C.: 
October 2000. 

• Researchers examined school and grade-level achievement data from 
40 Edison schools in eight states and compared it to data gathered from 
school districts and other schools. Weaknesses included insufficient 
information about the methodology employed by the states, including 
construction of comparison groups and matching techniques, and a 
lack of analysis of attrition rates. 

• Gomez. Ph.D., Joseph and SaUy Shay, Ph.D. Evaluation of the Edison 
Project School. Final Report, 1999-00 (portions related to parental 
satisfaction and involvement, and school climate). Office of Evaluation 
and Research, Miami-Dade County Public Schools (MDCPS), April 2001.® 

• Gomez and Shay examined responses from surveys MDCPS had 
administered to parents and teachers from both the Edison school and 
the control group. However, the outcomes related to parental 
satisfaction and involvement were measured with single-item survey 
questions that do not seem to capture the full context of the concepts. 
School climate was measured with a single-item question on a teacher 
survey and with school archival data.^ Shay and Gomez did not report 
whether any differences are statistically significant, in part because 



^Analyses of individual schools were included in an appendix provided upon request from 
the authors. We included this study in our review because it explicitly addressed the results 
of schools managed by the three companies in this report,- imhke other studies that did not 
disaggregate school results by company. 

^We only examined the portion of this study that relates to the effect of the Edison school 
on parental satisfaction and involvement and school climate. We examined the data and 
analysis that related to the academic achievement piece in Dr. Shay’s dissertation, A 
Longitudinal Study of Achievement Outcomes in a Privatized Public School: A G^'owth 
Curve Analysis, reported in the text. 

“'This includes data on students’ attendance rates, student mobility, indoor and outdoor 
suspensions, and the teacher-student ratio. 
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Appendix II: Studies and Other Information 
Sources Considered 



they acknowledged it would be inappropriate to conduct tests of 
significance on single-item questions. Therefore, there is no evidence to 
determine whether Edison school parents were more satisfied or 
involved than those in the control group, or whether the Edison school 
improved school climate. 

We are aware of other studies and reports that address the effect of 
Chancellor Beacon Academies, Mosaica Education, and Edison Schools on 
academic achievement, parental satisfaction, parental involvement, or 
school climate; however, the following are examples that did not meet the 
criteria for inclusion in our review. 

• District of Columbia Public Charter School Board. School Performance 
Reports. Washington, D.C.: August 2001. 

• Department of Research, Evaluation, and Assessment, Minneapolis 
Public Schools. Edison/PPL School Information Report 2000-2001 . 
Minneapolis, Mirmesota: 2001. 

• Department of Administration, Counseling, Educational and School 
Psychology, Wichita State University. An Independent Program 
Evaluation for the Dodge-Edison Partnership School: First Year 
Interim Report. Wichita, Kansas: 1996. 

• Missouri Department of Elementary and Secondary Education. Charter 
School Performance Study: Kansas City Charter Schools. Jefferson 
City, Missouri; 2001. 

• Company-provided information such as annual reports and school 
performance reports. 

Other sources of general information included school district websites and 
other educational services, such as Standard and Poor’s School Evaluation 
Services and the National Association of Charter School Authorizers’ 
Educational Service Provider Information Clearinghouse. 
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Appendix III: Comments from the 
Department of Education 



UNITED STATES DEPARTMENT OF EDUCATION 

THE DEPUTY SECRETARY 

October 8, 2002 

Ms. Mamie S. Shaul 

Director, Education, Workforce and Income Security Issues 
General Accounting Office 
Washington, DC 20548 

Dear Ms. Shaul; 

Thank you for the opportunity to review and comment on the draft report, “Public Schools: 
Insufficient Research to Determine Effectiveness of Selected Private Education Companies.” 
Across the country, we see how public education is being transformed by expanding the options 
and choices available to parents. For example, when the school year opened last month, there 
were over 2,700 charter schools operating in 39 States nationwide, an impressive feat 
considering the charter school movement is just a decade old. 

Charter schools, which are public schools of choice that operate free from many State and local 
district rules in exchange for heightened accountability, allow a variety of different groups to 
enter the public education market and manage schools. Some of these new managers include 
local community groups and businesses, as well as private education management organizations 
(EMOs), like those identified in your report. As you also indicate in your report, many of these 
EMOs also operate traditional public schools on a contract basis. Are these companies effective 
in boosting student achievement? Unfortunately, as your study details, there is insufficient data 
on the performance of EMOs due to the relatively small number of schools they manage and 
other methodological constraints. 

The only way to learn more about charter schools, alternate governance arrangements (such as 
schools managed by EMOs), and other reforms is through further experimentation and research. 
Those of us in public education should embrace these exciting new innovations that hold out the 
promise of boosting student achievement. Unfortunately, we know that too many of our schools 
are not performing at the levels they should, so we must explore alternatives while rigorously 
examining their results. Parents too must be free to choose which schools best suit their 
children’s needs. 

I applaud the work of States and local communities in broadening the choices available to 
parents and experimenting with new, innovative approaches to the delivery of public education. 
In the future, I know there will be a deeper body of research available and I encourage the GAO 
and other organizations to continue their evaluation efforts. Together, we will work to ensure 
that no child is left behind. 




Sincerely, 




400 MARYLAND AVE., S.W., WASHINGTON, D.C. 20202-0500 
www.ed.gov 

Our mission is to ensure equal access to education and to promote educational excellence throughout the Nation. 
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GAO’s Mission The General Accounting Office, the investigative arm of Congress, exists to 

support Congress in meeting its constitutional responsibilities and to help 
improve the performance and accountabihty of the federal government for the 
American people. GAO examines the use of public funds; evaluates federal 
programs and pohcies; and provides analyses, recommendations, and other 
assistance to help Congress make informed oversight, policy, and funding 
decisions. GAO’s commitment to good government is reflected in its core values 
of accountability, integrity, and reliability. 
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